I am testing a large program that reads a whole lot of data into an Array. I'm experiencing a "Memory Exhausted" error dialog with corresponding "An unexpected error has occurred" DXL error.
Have no idea how to proceed. Ideas?
I notice:
-
It occurs in the exact same line of code parsing the exact same Object each time; even when some hidden module's are closed.
-
That when I remove some of the captured columns of the Array, I get the same error at the same exact location in the code; parsing the same exact Object. When I bypass that exact Object or that row, it occurs in the very next Object or row.
-
When I bypass that module it occurs similarly in the next module.
-
It happens very fast, after about 8 seconds of run time.
-
I'm not creating a bunch of other data structures like Skip lists.
-
Windows System Memory is not that bad, 1.3gig
-
Closing the dialog frees about .4gig of system memory.
-
Get the same error at the same place even after closing other Windows.
-
Get the same error on the other computer.
-Louie
llandale - Tue Feb 14 09:42:27 EST 2012 |
|
Re: Memory Exhaused
Mathias Mamsch - Tue Feb 14 12:00:35 EST 2012
Hi Louie,
it seems pretty obvious that in your program somewhere is a memory leak or you are creating some serious large strings in your program. Those bugs can be hard to find, but what you experience is most likely explained as followed:
-
The memory leak happens somewhere in your program, but the memory exhaustion happens somewhere completely independent. The reason is, that where the leak occurs you just forget to free your data, but you do not allocate your data there. That means the program could freeze at any position, where you allocate some memory. That means the DXL backtrace will give you nothing.
-
Since the memory is leaked somewhere, it is not of interest what test data you use. The memory exhaustion will occur when your memory is full, independent of what allocation you are trying to do.
-
It is pretty hard to fill up memory by creating empty skip lists. This will more likely cause bad slowdown. Creating data structures therefore is most probably not the cause.
-
Since the memory is filled up pretty quickly, I would guess you are either creating lots of large strings (string append loop?) which can fill up memory most quickly.
-
When a memory leak occurs, the normal DXL cleanup routines might not be executed anymore. That means that DOORS usually will clean up memory when the DXL ends, but in case of an exception the interpreter really shuts down without cleaning up. Therefore part of your allocated memory might stay alive, even though your DXL ended.
Can you send source code? I have some means of finding memory leaks in a specially prepared environment.
Regards, Mathias
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS
|
|
Re: Memory Exhaused
llandale - Tue Feb 14 13:40:01 EST 2012 Mathias Mamsch - Tue Feb 14 12:00:35 EST 2012
Hi Louie,
it seems pretty obvious that in your program somewhere is a memory leak or you are creating some serious large strings in your program. Those bugs can be hard to find, but what you experience is most likely explained as followed:
-
The memory leak happens somewhere in your program, but the memory exhaustion happens somewhere completely independent. The reason is, that where the leak occurs you just forget to free your data, but you do not allocate your data there. That means the program could freeze at any position, where you allocate some memory. That means the DXL backtrace will give you nothing.
-
Since the memory is leaked somewhere, it is not of interest what test data you use. The memory exhaustion will occur when your memory is full, independent of what allocation you are trying to do.
-
It is pretty hard to fill up memory by creating empty skip lists. This will more likely cause bad slowdown. Creating data structures therefore is most probably not the cause.
-
Since the memory is filled up pretty quickly, I would guess you are either creating lots of large strings (string append loop?) which can fill up memory most quickly.
-
When a memory leak occurs, the normal DXL cleanup routines might not be executed anymore. That means that DOORS usually will clean up memory when the DXL ends, but in case of an exception the interpreter really shuts down without cleaning up. Therefore part of your allocated memory might stay alive, even though your DXL ended.
Can you send source code? I have some means of finding memory leaks in a specially prepared environment.
Regards, Mathias
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS
While the bulk of my code accesses the array with X-coord = Columns and Y-coord = Rows, and using put(Array, value, col#, row#) my initialization routine was setting put(Array, value, row#, col#); I thus seem to be creating an array with 5392 x 5392 = 29,073,664 cells (5392 being consistently the row triggering the error).
Not sure what that says about the internal workings of DOORS, or the internal workings of my brain, but perhaps someone can comment on the former.
-Louie
So I guess for an Array with 5 columns we have a max # rows of about 5.8 million.
|
|
Re: Memory Exhaused
Mathias Mamsch - Tue Feb 14 13:45:29 EST 2012 llandale - Tue Feb 14 13:40:01 EST 2012
Doh! Sometimes I wonder how I put my shoes on in the morning; explaining why I got velcro straps instead of laces.
While the bulk of my code accesses the array with X-coord = Columns and Y-coord = Rows, and using put(Array, value, col#, row#) my initialization routine was setting put(Array, value, row#, col#); I thus seem to be creating an array with 5392 x 5392 = 29,073,664 cells (5392 being consistently the row triggering the error).
Not sure what that says about the internal workings of DOORS, or the internal workings of my brain, but perhaps someone can comment on the former.
-Louie
So I guess for an Array with 5 columns we have a max # rows of about 5.8 million.
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS
|
|
Re: Memory Exhaused
llandale - Tue Feb 14 15:53:40 EST 2012 Mathias Mamsch - Tue Feb 14 13:45:29 EST 2012
Doh ... Well I guess then my suggestion was plainly wrong. Well lets hope that some day I will find a problem for my solution ;-) Regards, Mathias
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS
I'm ignorant here, but I've notice in the past that the following things SLOW down DOORS drastically; I've never actually had such a "Memory low" error before:
-
Too many module's open, typically invisibly
-
Too many data structures created (Skips/Buffers embedded within Skip Lists)
I'm surprised that I could not SEE that DOORS was eating up too much Memory, using Task Manager.
Whole thing is curious, don't know what to think about it.
-Louie
|
|
Re: Memory Exhaused
Mathias Mamsch - Wed Feb 15 11:39:45 EST 2012 llandale - Tue Feb 14 15:53:40 EST 2012
Don't worry, I've gone through your list myself numerous times...
I'm ignorant here, but I've notice in the past that the following things SLOW down DOORS drastically; I've never actually had such a "Memory low" error before:
-
Too many module's open, typically invisibly
-
Too many data structures created (Skips/Buffers embedded within Skip Lists)
I'm surprised that I could not SEE that DOORS was eating up too much Memory, using Task Manager.
Whole thing is curious, don't know what to think about it.
-Louie
Mathias Mamsch, IT-QBase GmbH, Consultant for Requirement Engineering and D00RS
|
|